Skip to content

Daily sync merges upstream's newest nightly tag through a pull request - #38

Merged
tomkaczocha merged 15 commits into
mainfrom
t3/factory-issue-4
Oct 15, 2026
Merged

tomkaczocha merged 15 commits into
mainfrom
t3/factory-issue-4

Conversation

@rawphp

@rawphp rawphp commented Oct 15, 2026 •

Copy link
Copy Markdown

Closes #4

Contract

  • done: factory/scripts/sync.sh picks upstream's newest v*-nightly.* tag by version order and prints up to date (exit 0) when factory/UPSTREAM names it; otherwise merges it on sync/<tag> from origin/main with --no-ff, regenerates apps/web/src/routeTree.gen.ts when only that file conflicts, records <tag> <sha>, commits and runs check.sh. Green: push, open sync: <tag>, merge with a merge commit. Red: push, draft PR, issue Sync to <tag> fails checks (factory:proposed) with the failing step and its last 200 lines. Any other conflict: abort, push nothing, file or comment on Sync to <tag> conflicts, exit 1. factory-sync.yml runs it daily at 03:17 UTC and on dispatch.
  • out: Linear; release build; deleting sync branches; disabling upstream workflows; a second run on a tag that already has a sync branch (its push fails non-fast-forward and the run goes red).
  • risk: ci-automerge. GITHUB_TOKEN lands a green sync on main with no human. vp i, the route generator and check.sh run upstream's code; the script drops GH_TOKEN from their environment and the checkout no longer persists the token, which is hygiene, not a boundary (same runner user).
  • test: vp test run apps/server/src/factory/sync.test.ts → pass (10 tests)

Known gap

  • GITHUB_TOKEN cannot push changes under .github/workflows/, and no permissions: key grants it. When a nightly touches upstream's workflows, the push fails, the script prints sync: push of sync/<tag> failed; <tag> is not synced and exits 1, and the run is red with no issue filed. The issue pins GITHUB_TOKEN, so this build does not swap tokens. The one-line fix, if wanted, is a fine-grained PAT or App token with the workflows permission as the step's GH_TOKEN; PRs opened with it would then trigger factory-ci, which is harmless.
  • gh auth setup-git with GH_TOKEN and persist-credentials: false is the documented pattern but was not exercised in CI from this run. The first workflow_dispatch run proves it.

Acceptance criteria

  • Newest v*-nightly.* by version order; up to date exit 0 when UPSTREAM names it — picks the newest nightly tag by version order and stops when UPSTREAM names it; live run against the real upstream printed newest nightly: v0.0.46-nightly.20261015.3145 then up to date
  • sync/<tag> from origin/main, merge --no-ff, UPSTREAM written, commit, check.sh — merges the newest tag, records it, runs the checks and merges the pull request (merge commit with two parents, UPSTREAM = <tag> <sha>)
  • Route tree conflict regenerated with upstream's generator — regenerates the route tree when only that file conflicts; factory/scripts/regenerate-routes.sh verified on the real tree: from a corrupted file it reproduces the committed routeTree.gen.ts byte for byte
  • Other conflict: abort, push nothing, issue Sync to <tag> conflicts listing paths, exit 1; comment when open — aborts on any other conflict..., aborts on a file the fork edited and the tag deleted, comments on the open conflicts issue instead of opening another
  • Green: push, PR sync: <tag>, merge commit — merges the newest tag... (gh pr merge <url> --merge)
  • Red: push, draft PR, issue Sync to <tag> fails checks labelled factory:proposed with failing step and last 200 lines — failing checks open a draft pull request and a factory:proposed issue with the failing step
  • factory-sync.yml: cron 17 3 * * * and workflow_dispatch, ubuntu-latest, contents: write, pull-requests: write, issues: write — verified .github/workflows/factory-sync.yml:4-11,21
  • Test against a local fake upstream with three nightly tags — sync.test.ts builds v0.0.9-nightly.20261013.1, v0.0.10-nightly.20261014.2950, v0.0.10-nightly.20261015.3145 plus a release tag v0.0.10 the pattern must skip

Decisions the issue left open

  • The conflicts issue carries no label (the issue names factory:proposed only for the fails-checks issue).
  • Failing checks exit 1 after filing the issue, so both failure paths show a red run.
  • Tags must match ^v[0-9]+\.[0-9]+\.[0-9]+-nightly\.[0-9]+\.[0-9]+$ before they reach git, gh or a branch name; an upstream rename of the nightly scheme fails loudly.
  • vp i runs after the merge so the generator and checks see the merged lockfile.
  • regenerate-routes.sh is its own script so a person resolving a seam conflict by hand runs the same generator.

Gate

  • vp test run apps/server/src/factory → exit 0 (5 files, 24 tests)
  • vp run --filter t3 typecheck → exit 0
  • extra: factory/scripts/check-seams.sh → exit 0

Models

  • maker: this session, claude-fable-5-1 (.dw/config.yml has no models block; the roster's sonnet maker was not used, the orchestrator made the change itself)
  • checker: claude-opus-5-5 (host subagents)

Review

  • cycles: 2, then a small final batch read by the orchestrator
  • fixed: cycle 1 (correctness, reliability, security): a merge failing without unmerged paths fell through as a route tree conflict and committed the UPSTREAM bump alone; a refused push died without its own message; GH_TOKEN reached vp i and check.sh and sat in .git/config; the tag reached a jq program unescaped; failing checks exited 0. Cycle 2 (correctness, testing with 12 mutants, 11 killed): the token comment overstated env -u as isolation; the surviving mutant (--diff-filter=M) gets a modify/delete conflict test; the gh stub re-implemented the title filter, so the exact match moved into the script and the stub prints raw rows.
  • leftovers: none

Lessons

  • A detached review copy of a pnpm workspace needs per-package links. Symptom: the testing reviewer's baseline run failed before any test ran. Cause: the brief linked only the root node_modules; vitest also needs apps/<pkg>/node_modules and the shared packages' node_modules, and the relative path in the brief was one level short. Default: tell the reviewer to link whatever the baseline run asks for from the maker's install, run the suite green once, and only then mutate.
  • A criterion that pins a mechanism a platform rule breaks is a receipt gap, not a guess or a stop. Symptom: the issue pinned GITHUB_TOKEN, which cannot push workflow-file changes, and the sync merges a repo that changes its workflows weekly. Cause: the author did not know the limit. Default: verify the rule once (docs or one search), build as written, make that path fail with a message that names the branch and tag, and put the gap and its one-line fix at the top of the receipt.
  • On a fork, gh pr create without --repo opens the pull request on the parent. Symptom: the draft landed on pingdotgg/t3code (Daily sync merges upstream's newest nightly tag through a pull request pingdotgg/t3code#19873) and had to be closed with an apology before being opened on the fork. Cause: gh resolves the base repository of a fork to its parent by default. Default: every gh pr create and gh pr edit in Ship passes --repo <owner/repo> from the issue, and Ship checks the printed URL's repository before linking it.
  • A stub that re-implements the filter the script passes to a CLI tests the stub. Symptom: the mutation review found the gh --jq title match untested. Default: move the decision into the script (shell match on raw rows) and have the stub print rows, so the script's own logic is what the test exercises.

Commits

  • 92ae186bc test(factory): sync.sh picks the newest nightly tag and merges it through a pull request — Make slice, red
  • dc320ebb7 feat(factory): daily sync merges upstream's newest nightly tag through a pull request — Make slice
  • d2ce21d50 docs(factory): describe the daily upstream sync — Make slice (docs, no red phase)
  • d9f006df6 test(factory): sync.sh keeps the token from upstream code and stops on a failed merge or push — Check cycle 1, red
  • 11c6cfdcc fix(factory): sync.sh stops on a merge or push failure and keeps the token from upstream code — Check cycle 1
  • 9b077e384 test(factory): sync.sh matches the conflicts issue title itself; modify/delete conflicts abort — Check cycle 2, red for the title match; the modify/delete test passed already
  • 159886a61 fix(factory): sync.sh matches the conflicts issue title in shell, not in jq — Check cycle 2

Files

  • factory/scripts/sync.sh — the sync
  • factory/scripts/regenerate-routes.sh — runs the web app's TanStack generator standalone
  • .github/workflows/factory-sync.yml — daily 03:17 UTC and dispatch
  • apps/server/src/factory/sync.test.ts — fake upstream, fork and bare origin; gh, vp, check.sh stubbed
  • factory/README.md — sync section

Run notes

…ough a pull request

Red: factory/scripts/sync.sh does not exist yet, so every case fails on
reading it. Covers the tag choice by version order, the up-to-date exit,
a clean merge that records UPSTREAM and merges the pull request, a route
tree conflict that regenerates, another conflict that aborts and files or
comments on an issue, and failing checks that open a draft and an issue.

Issue: #4
…h a pull request

sync.sh picks the newest v*-nightly.* tag by version order, stops with
"up to date" when factory/UPSTREAM names it, and otherwise merges it on
sync/<tag> with --no-ff, regenerating the route tree with
regenerate-routes.sh when only that file conflicts. Any other conflict
aborts and files or comments on `Sync to <tag> conflicts`. Green checks
push, open `sync: <tag>` and merge it; red checks open it as a draft and
file `Sync to <tag> fails checks` with the failing step's last 200 lines.
factory-sync.yml runs it daily at 03:17 UTC and on dispatch.

The test's gh stub answers `issue list` by title so the comment-or-create
decision is covered; the conflicts body may mention the route tree in its
hint, so the test checks the list line only.

Issue: #4
…n a failed merge or push

Red: a merge that fails without a conflict falls through as a route tree
conflict, a refused push dies without saying why, a tag with a quote is
accepted, vp i and check.sh see GH_TOKEN, and failing checks exit 0.

Issue: #4
…token from upstream code

A merge that failed without unmerged paths was taken for a route tree
conflict and went on to commit the UPSTREAM bump alone; it now prints
git's output and exits 1. A refused push says which branch and tag. The
tag must match the nightly pattern before it reaches gh, git or jq.
vp i, the generator and check.sh run without GH_TOKEN; the workflow
stops persisting the checkout token and routes git push through gh's
credential helper instead. Failing checks exit 1 after filing the issue,
so the scheduled run is red on both failure paths.

Issue: #4
…fy/delete conflicts abort

Red: the gh stub now prints every open issue as number and title rows,
so the comment-or-create decision is the script's own match; a fuzzy
search hit listed first must not win. The modify/delete case passes
already and pins --diff-filter=U.

Issue: #4
… in jq

The tag no longer reaches a jq program, and the exact-title match is the
script's own, so a fuzzy search hit cannot receive the comment. The
token comments say what env -u does and does not do.

Issue: #4
@tomkaczocha
tomkaczocha marked this pull request as ready for review October 15, 2026 12:10
@rawphp
rawphp marked this pull request as draft October 15, 2026 12:14
@rawphp
rawphp marked this pull request as ready for review October 15, 2026 12:21
@rawphp

rawphp commented Oct 15, 2026

Copy link
Copy Markdown
Author

Factory gate: passed (attempt 2, run by the factory in a fresh checkout of the pull request's head 5db9a2a merged with the base branch tip aeb0727)

Command Exit Time
factory/scripts/check.sh 0 27.8 s

@rawphp

rawphp commented Oct 15, 2026

Copy link
Copy Markdown
Author

Factory gate: passed again (attempt 3, run by the factory in a fresh checkout of the pull request's head 5db9a2a merged with the base branch tip 952129a)

Command Exit Time
factory/scripts/check.sh 0 22.8 s

… one issue per problem

- gh calls run with GH_REPO from origin's GitHub URL, so a local run cannot open
  pull requests or issues on the fork's parent; a non-GitHub origin stops first.
- A failing `vp i` or route generator aborts the merge, pushes nothing and files
  `Sync to <tag> fails checks` with its output, instead of dying silently.
- A refused push files `Sync to <tag> cannot be pushed`, naming the workflows
  token limit.
- A tag whose sync/<tag> branch is already on origin is left alone with exit 0,
  instead of failing non-fast-forward every night.
- Every issue goes through file_issue, which comments on an open issue of the
  same kind for any nightly tag and retitles it, so a lasting conflict or
  failure keeps one issue.
- regenerate-routes.sh refuses when vite.config.ts passes the router plugin
  other options; a test runs the real generator on a fixture tree.
- Tests: a release tag newer than every nightly, the issue search arguments.
…s silently

- A green sync fast-forwards main to the checked merge commit instead of asking
  GitHub to merge the branch into whatever main is by then; the push is refused
  when main moved during the checks, and the pull request then stays open with
  a comment for a person (exit 1).
- A sync branch already on origin waits on its open pull request (exit 0); with
  none open, the run that pushed it stopped early or the pull request was
  closed, so it files `Sync to <tag> is stuck` (exit 1) instead of passing.
…main's UPSTREAM

- When main cannot be fast-forwarded to the checked sync branch, the run files
  `Sync to <tag> cannot land` with the pull request, so the later runs that wait
  on that pull request are not the only trace.
- `up to date` compared the tag with the checkout's factory/UPSTREAM, which an
  earlier local run leaves on the sync branch; it now reads origin/main's. The
  second-run test caught it.
…d run after a stop

Covers both directions of a checkout whose UPSTREAM disagrees with origin's main, and what the next run does after failing checks (waits on the draft) and after a refused push (tries again, comments on the open issue).
@rawphp

rawphp commented Oct 15, 2026

Copy link
Copy Markdown
Author

Factory gate: passed again (attempt 4, run by the factory in a fresh checkout of the pull request's head b919a6f merged with the base branch tip 952129a)

Command Exit Time
factory/scripts/check.sh 0 21.3 s

factory-sync.yml now runs sync.sh as two jobs. `prepare` has contents: read
and no token: it merges, runs `vp i`, the route generator and check.sh, and
uploads the outcome and a git bundle of sync/<tag>. `publish` holds
GITHUB_TOKEN and runs only git and gh: push, the fast-forward landing, pull
requests and issues, including the issue for a refused push.

publish treats the artifact as untrusted. It validates the outcome and the
tag, and refuses a bundled branch that is not a merge of a commit on main and
the tag it fetches itself, or whose tree differs from `git merge-tree` of the
two in anything but factory/UPSTREAM (which must name the tag) and a
conflicted route tree. The prepare job saves no cache, since it runs upstream's
install scripts. With no arguments, sync.sh still runs both phases from one
checkout.
…pare's word

- Tests for a bundled merge whose first parent is not on main, one with a
  third parent, and one that commits git's own conflicted tree; each guard in
  verify_candidate now fails a test when removed.
- The script header and README no longer claim a compromised prepare job can
  change nothing else: a regenerated route tree's content is its word too,
  like the check verdict, since checking either means running upstream's code.
…landing

When a tag conflicts only in apps/web/src/routeTree.gen.ts, the regenerated
file comes from the unprivileged job and nothing can check it without running
upstream's generator. publish now pushes the branch and opens `sync: <tag>`,
files `Sync to <tag> needs a route-tree review` with the pull request, exits 1,
and never fast-forwards main; later runs wait on that pull request.
@tomkaczocha
tomkaczocha merged commit 465a1e5 into main Oct 15, 2026
@rawphp
rawphp deleted the t3/factory-issue-4 branch October 28, 2026 01:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Daily sync merges upstream's newest nightly tag through a pull request

2 participants